Deep Reinforcement Learning for Solving Vehicle Routing Problems With Backhauls.

The vehicle routing problem with backhauls (VRPBs) is a challenging problem commonly studied in computer science and operations research. Featured by linehaul (or delivery) and backhaul (or pickup) customers, the VRPB has broad applications in real-world logistics. In this article, we propose a neural heuristic based on deep reinforcement learning (DRL) to solve the traditional and improved VRPB variants, with an encoder-decoder structured policy network trained to sequentially construct the routes for vehicles. Specifically, we first describe the VRPB based on a graph and cast the solution construction as a Markov decision process (MDP). Then, to identify the relationship among the nodes (i.e., linehaul and backhaul customers, and the depot), we design a two-stage attention-based encoder, including a self-attention and a heterogeneous attention for each stage, which could yield more informative representations of the nodes so as to deliver high-quality solutions. The evaluation on the two VRPB variants reveals that, our neural heuristic performs favorably against both the conventional and neural heuristic baselines on randomly generated instances and benchmark instances. Moreover, the trained policy network exhibits a desirable capability of generalization to various problem sizes and distributions.

Conditional Neural Heuristic for Multiobjective Vehicle Routing Problems.

Existing neural heuristics for multiobjective vehicle routing problems (MOVRPs) are primarily conditioned on instance context, which failed to appropriately exploit preference and problem size, thus holding back the performance. To thoroughly unleash the potential, we propose a novel conditional neural heuristic (CNH) that fully leverages the instance context, preference, and size with an encoder-decoder structured policy network. Particularly, in our CNH, we design a dual-attention-based encoder to relate preferences and instance contexts, so as to better capture their joint effect on approximating the exact Pareto front (PF). We also design a size-aware decoder based on the sinusoidal encoding to explicitly incorporate the problem size into the embedding, so that a single trained model could better solve instances of various scales. Besides, we customize the REINFORCE algorithm to train the neural heuristic by leveraging stochastic preferences (SPs), which further enhances the training performance. Extensive experimental results on random and benchmark instances reveal that our CNH could achieve favorable approximation to the whole PF with higher hypervolume (HV) and lower optimality gap (Gap) than those of the existing neural and conventional heuristics. More importantly, a single trained model of our CNH can outperform other neural heuristics that are exclusively trained on each size. In addition, the effectiveness of the key designs is also verified through ablation studies.

Regulation of Sirtuins in Sepsis-Induced Myocardial Damage: The Underlying Mechanisms for Cardioprotection.

Sepsis is defined as "a life-threatening organ dysfunction caused by a dysregulated host response to infection". Although the treatment of sepsis has evolved rapidly in the last few years, the morbidity and mortality of sepsis in clinical treatment are still climbing. Sirtuins (SIRTs) are a highly conserved family of histone deacetylation involved in energy metabolism. There are many mechanisms of sepsis-induced myocardial damage, and more and more evidence show that SIRTs play a vital role in the occurrence and development of sepsis-induced myocardial damage, including the regulation of sepsis inflammation, oxidative stress and metabolic signals. This review describes our understanding of the molecular mechanisms and pathophysiology of sepsis-induced myocardial damage, with a focus on disrupted SIRTs regulation. In addition, this review also describes the research status of related therapeutic drugs, so as to provide reference for the treatment of sepsis.

NAD+-A Hub of Energy Metabolism in Heart Failure.

Heart failure is a condition where reduced levels of adenosine triphosphate (ATP) affect energy supply in myocardial cells. Nicotinamide adenine dinucleotide (NAD+) plays a crucial role as a coenzyme for electron transfer in energy metabolism. Decreased NAD+ levels in myocardial cells lead to inadequate ATP production and increased susceptibility to heart failure. Researchers are exploring ways to increase NAD+ levels to alleviate heart failure. Targets such as sirtuin2 (sirt2), sirtuin3 (sirt3), Poly (ADP-ribose) polymerase (PARP), and diastolic regulatory proteins are being investigated. NAD+ supplementation has shown promise, even in heart failure with preserved ejection fraction (HFpEF). By focusing on NAD+ as a central component of energy metabolism, it is possible to improve myocardial activity, heart function, and address energy deficiency in heart failure.

Learning Feature Embedding Refiner for Solving Vehicle Routing Problems.

While the encoder-decoder structure is widely used in the recent neural construction methods for learning to solve vehicle routing problems (VRPs), they are less effective in searching solutions due to deterministic feature embeddings and deterministic probability distributions. In this article, we propose the feature embedding refiner (FER) with a novel and generic encoder-refiner-decoder structure to boost the existing encoder-decoder structured deep models. It is model-agnostic that the encoder and the decoder can be from any pretrained neural construction method. Regarding the introduced refiner network, we design its architecture by combining the standard gated recurrent units (GRU) cell with two new layers, i.e., an accumulated graph attention (AGA) layer and a gated nonlinear (GNL) layer. The former extracts dynamic graph topological information of historical solutions stored in a diversified solution pool to generate aggregated pool embeddings that are further improved by the GRU, and the latter adaptively refines the feature embeddings from the encoder with the guidance of the improved pool embeddings. To this end, our FER allows current neural construction methods to not only iteratively refine the feature embeddings for boarder search range but also dynamically update the probability distributions for more diverse search. We apply FER to two prevailing neural construction methods including attention model (AM) and policy optimization with multiple optima (POMO) to solve the traveling salesman problem (TSP) and the capacitated VRP (CVRP). Experimental results show that our method achieves lower gaps and better generalization than the original ones and also exhibits competitive performance to the state-of-the-art neural improvement methods.

Learning Improvement Heuristics for Solving Routing Problems.

Recent studies in using deep learning (DL) to solve routing problems focus on construction heuristics, whose solutions are still far from optimality. Improvement heuristics have great potential to narrow this gap by iteratively refining a solution. However, classic improvement heuristics are all guided by handcrafted rules that may limit their performance. In this article, we propose a deep reinforcement learning framework to learn the improvement heuristics for routing problems. We design a self-attention-based deep architecture as the policy network to guide the selection of the next solution. We apply our method to two important routing problems, i.e., the traveling salesman problem (TSP) and the capacitated vehicle routing problem (CVRP). Experiments show that our method outperforms state-of-the-art DL-based approaches. The learned policies are more effective than the traditional handcrafted ones and can be further enhanced by simple diversifying strategies. Moreover, the policies generalize well to different problem sizes, initial solutions, and even real-world data set.